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This  slide  describes  the  overall  methodology  employed  to  develop  this  dataset.  Aggregated 
data  downloads  from  FPDS  are  unavailable  for  pre-2000  data,  requiring  a  different  approach. 

•  The  data  set  of  approximately  six  million  entries  includes  all  contracts  in  the  Federal 
Procurement  Data  System  (FPDS)  completed  between  FY2007  and  FY2013. 

•  Federal  regulations  require  only  that  all  unclassified  prime  contracts  worth  $2,500  and  above 
be  reported  to  FPDS. 

•  FPDS  data  are  constantly  being  updated,  including  those  for  back  years.  As  a  consequence, 
the  dollar  totals  for  a  given  year  may  have  changed  since  the  data  was  downloaded. 

•  All  dollar  figures  are  in  current  dollars  because  contract  ceilings  do  not  account  for  inflation. 
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Dependent  Variable:  Number  of  offers 
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Number  of  Partially  or  Completely 
Terminated  Contracts 


Dependent  Variable:  Terminations 
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Dependent  Variable:  Number  of  Change  Orders 
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Dependent  Variable:  Cost  Ceiling-Raising  Change  Orders 
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Introduction  to  Bayesian  Networks 

•  The  model  is  based  on  Bayesian  inferential  statistics,  and  relates  the  odds  of  an  event  (Event  Al) 
to  the  odds  of  another  event  (Event  A2)  before  and  after  the  occurrence  of  some  other  event 
(Event  B). 

•  It  is  a  way  of  representing  a  set  of  variables  nodes  and  their  conditional  dependencies  via  a 
directed  acyclic  graph  (DAG). 

•  Because  a  Bayesian  network  is  a  complete  model  for  variables  and  their  relationships,  it  can  be 
used  to  answer  probabilistic  queries  about  them. 

•  The  model  is  built  in  the  open  source  statistical  programming  language  R  using  two  modules. 

The  module  BnLearn  is  used  for  the  Bayesian  network  learning  process,  which  turns  the 
collected  data  into  a  DAG. 

•  The  module  gRain  is  used  for  the  second  part  of  the  process,  creating  the  conditional  probability 
table  and  then  querying  the  resulting  multiples. 

•  The  data  and  the  open  source  processing  and  analytic  programming  code  used  to  implement 
this  process  are  available  through  the  CSISdefense  fixed  price  GitHub  repository 
(Available  at:  https://github.com/CSISdefense/Fixed-price) 
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White  List:  Arcs  which  are  required  to  present  in  the  Bayesian  model 
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Black  List:  Arcs  which  are  not  allowed  in  the  Bayesian  model 
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Resulting  Bayesian  Model 
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Hypothesis  1:  Large  R&D  contracts  will  perform  better  as  cost-based 
contracts 
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Support  in  Literature:  Goel  (1999)  found  "that  the  [contracting  office]  principal  prefers  a  cost-plus 
contract  in  cases  of  large  R&D  projects  or  rising  innovation  benefits...  The  agent  increases  its 
research  out-lays  in  response  to  a  higher  sharing  rate  when  the  expected  rewards  from 
innovation  significantly  exceed  research  costs."  Similarly  Kendall  (2015)  describes  low  technical 
risk  as  a  reason  to  choose  fixed  price  contracts.  Large  R&D  contracts  are  known  for  their 
technical  risk. 

Results: 

•  Number  of  Offers:  The  hypothesis  was  supported,  cost-based  contracts  had  a  substantially 
lower  single  offer  competition  rate,  22.5  percent  for  cost-based  versus  29.7  percent  for  fixed 
price.  Cost-based  similarly  had  a  higher  rate  of  competition  with  five  or  more  offer.  This  pattern 
held  for  long  duration  contracts  and  for  aircraft  contracts.  In  the  latter  case  fixed  price  contracts 
received  only  a  single  offer  37.6  percent  of  the  time  versus  11.3  percent  for  cost-based !  Cost- 
plus  only  loses  its  advantage  for  IDV  contracts. 
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Hypothesis  2:  Complex  projects,  as  measured  by  pre-milestone  B 
major  defense  acquisition  project  status,  will  perform  better  as 
cost-based  contracts 

Support  in  Literature:  Bajari  and  Tadelis  (2001)  found  that  cost-plus  contracts  are  preferred  to 
fixed  price  contracts  when  a  project  is  more  complex  (Bajari  and  Tadelis  2001). 

Results: 

Number  of  offers:  The  hypothesis  was  weakly  supported  using  project  interlinkages  as  a  proxy  for 
complexity.  However,  the  results  were  not  highly  robust  and  were  contradicted  for  two  of  the 
five  controls.  The  single  offer  competition  rate  for  cost-plus  contracts  was  six  percentage  points 
lower  for  large  contracts  and  two  percentage  points  lower  for  aircraft.  Since  both  large 
contracts  and  aircraft  and  drone  contracts  are  associated  with  MDAPs,  this  suggests  that  the 
study  team  will  need  to  refine  linkages  or  test  the  hypothesis  directly  by  looking  at  system 
equipment  codes. 
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Hypothesis  3:  Contracts  with  a  longer  duration  will  perform 
better  as  cost-based  contracts 
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Support  in  Literature:  Braucher  (1953)  found  that  "price  redetermination  might  be  used  whenever 
contingency  charges  otherwise  would  be  included  in  a  contract  price  due  to  such  factors  as 
prolonged  delivery  schedules,  unstable  market  conditions  for  material  or  labor,  or  uncertainty 
as  to  cost  of  performance." 

Results: 

•  Number  of  Offers:  The  hypothesized  relationship  did  not  hold.  Unlike  Hypothesis  2,  the  results 
appeared  to  be  fairly  robust.  Overall,  25.3  percent  of  fixed  price  contracts  with  durations  greater 
than  a  year  receive  only  one  offer  compared  to  31.4  percent  of  cost-plus  contracts.  This  gap 
narrows  to  less  than  a  percentage  point  for  large  contracts  and  aircraft  and  drone  contracts. 
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Hypothesis  4:  The  potential  for  greater  competition  improves 
fixed  price  performance 
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Support  in  Literature:  Goel  (2001)  argues  that  "[Government  principals]  would  prefer  a  fixed-price 
contract  when  the  number  of  bidders  increases."  The  hypothesis  that  fixed  price  contests  are 
preferred  by  acquisition  officials  when  they  are  likely  to  receive  more  competition  may  indicate 
that  contracts  that  are  more  likely  to  be  competed  will  perform  better. 

Results: 

•  Number  of  Offers:  This  hypothesis  is  not  testable  with  this  dependent  variable.  However,  given 
the  high  variability  for  the  Number  of  Offers  Received  for  fixed  price  contracts  and  the  number 
of  evidence  nodes  influencing  both  competition  and  number  of  offers,  this  hypothesis  should  be 
straightforward  to  test  in  future  stages. 
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Hypothesis  5:  Large  software  projects  perform  better  as  fixed 
price  contracts 

Support  in  Literature:  Gopal  and  Sivaramakrishnan  (2006)  studied  whether  "the  vendor's  ability  to 
leverage  information  asymmetry  about  capabilities  and  experiences  translates  into  the  vendor 
preferring  Fixed-Price  contract  to  secure  larger  information  rents"  and  found  support  for  "larger 
and  longer  projects  with  larger  teams."  For  this  iteration,  Electronics  and  Communications 
services  were  used  as  a  proxy  for  software. 

Results: 

•  Number  of  Offers:  Our  results  surrounding  this  hypothesis  were  inconclusive.  We  found  that  for 
small  contracts,  the  hypothesis  holds,  with  fixed  price  contracts  nearly  ten  percentage  points 
less  likely  to  receive  one  offer  compared  to  cost-based  (28.8  percent  versus  38.7  percent). 
However,  this  relationship  reverses  itself  for  large  contracts.  29.0  percent  of  fixed  price 
contracts  receive  only  a  single  offer  versus  21.2  percent  of  cost-based  contracts. 
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Releasing  our  data  into  the  wild: 
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Our  work  in  progress  is  available  at  http://www.GitHub.com/CSISdefense/Fixed-Price 
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